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ABSTRACT 

The Appalachian Rural Systemic Initiative (ARSI) is a 
collaborative mathematics, science, and technology education reform effort 
among six states in central Appalachia — Kentucky, North Carolina, Ohio, 
Tennessee, Virginia, and West Virginia. The project aims to stimulate 
sustainable systemic improvements in these subjects for K-12 students in a 
historically low-performing region. The ARSI model is based on a team approach 
to school reform. The district team consists of the ARSI district liaison, 
teacher-partner, school principals, and district superintendent. District 
teams are supported by ARSI resource collaboratives located at five 
universities in Appalachia. A key component of the model is the teacher- 
partner, a skilled science or mathematics teacher, selected by the school to 
provide leadership for reform efforts. Teacher-partners receive training from 
the resource collaboratives and are provided release time for mentoring other 
teachers, assisting with curriculum development, acquiring instructional 
resources, and generally providing leadership for program improvement. 

Catalyst schools are designed as incubator sites for implementation of 
standards-based practices and become models for other schools. The ARSI model 
employs a cyclic system beginning with identification and analysis of science 
and mathematics program needs followed by school and district planning, 
implementation of reform strategies, and evaluative feedback. Positive results 
during year 5 included leadership development, standards-based curriculum 
development, development of consistent policies, increased community 
engagement, professional development, and improved student achievement. 
(Contains 36 figures detailing improvements in these areas.) (TD) 



Reproductions supplied by EDRS are the best that can be made 
from the original document. 
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ARSI PROJECT OVERVIEW 



The Appalachian Rural Systemic Initiative (ARSI) is a collaborative mathematics, science and technology education reform 
effort among six states in Central Appalachia - Kentucky, North Carolina, Ohio, Tennessee, Virginia, and West Virginia. 
The overarching goal of the project is to stimulate sustainable systemic improvements in mathematics, science, and 
technology education for K-12 students in a region in which achievement has historically lagged behind other regions of the 
participating states. ARSI has made great strides toward achieving this goal by assisting schools create effective teaching 
and learning environments, align curricula with state and national standards, converge resources from local, state, and federal 
sources, and develop strong community partnerships focusing on mathematics, science, and technology education. 

The “ARSI Model” is based on a team approach to school reform. The district team consists of the ARSI District Liaison, 
Teacher Partner, school principals, and the district superintendent. District teams are supported by ARSI Resource 
Collaboratives located at five universities in the Appalachian region. A key component of the model is the Teacher Partner , 
a skilled science or mathematics teacher, selected by the school to provide leadership for reform efforts. Teacher Partners are 
provided release time for mentoring other teachers, assisting with curriculum development, acquiring of instructional 
resources, and generally providing leadership for program improvement efforts. The Teacher Partners have received 
extensive training and support from the ARSI Resource Collaboratives and are now providing leadership for district-wide 
reforms in science, mathematics, and technology education. Catalyst schools are designed as “incubator sites” for 
implementation of standards-based practices and are used as models for other schools in the district and region. The “ARSI 
Systemic Change Model” employs a cyclic system beginning with identification and analysis of science and mathematics 
program needs followed by school and district action planning, implementation of reform strategies, and evaluative feedback. 

The primary ARSI agents for support of district efforts are the ARSI Resource Collaboratives. Through the collaboratives, 
ARSI establishes a broad-based system that facilitates local planning and decision-making and creates collaborations that 
align vital processes and resources. The professional staff, the Collaborative Coordinators, serve as “field agents” for the 
ARSI project, providing direct services to districts as well as being responsible for identifying resources at the sponsoring 
universities and other state and regional agencies for professional development and technical assistance. ARSI continues to 
institutionalize the functions of the Resource Collaborative at each university so that these sites will continue as centers for 
science and mathematics education reform beyond the scope of NSF support. 

ARSI has established a broad-based system that facilitates local planning and decision-making by allowing students, 
teachers, administrators, and community members equitable access to factors critical to achieving the vision of quality 
mathematics, science, and technology education. 

As a result of ARSI’s efforts during the first five years of the project, student achievement has increased in science and 
mathematics in the catalyst schools, new and consistent policies increasing the opportunities for and rigor of mathematics and 
science programs have been enacted, and broad-based support for science and mathematics education has been achieved. 
These systemic improvements will contribute to the long-range goal of the ARSI project, to prepare a more competitive 
workforce not bound by its geography. 

L THE ARSI MODEL 

The principal goal of the Appalachian Rural Systemic Initiative continues to be accelerating improvements in student 
performance in science, mathematics, and technology throughout the central Appalachian region. ARSI has addressed three 
significant strategic goals during year 5: 

• Develop the knowledge and skills among K-12 teachers to create effective learning environments in which all 
students learn mathematics and science and use technology. 

• Develop a sustainable system providing students and teachers with timely, coordinated access to educational 
resources and services that support active, standards-based teaching and learning. 

• Develop the school leadership, regional partnerships, community involvement, and stakeholder support 
necessary to sustain long-term educational improvements. 

ARSI has demonstrated significant progress toward these goals during the tenure of the award. 
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Figure 1 

ARSI Systemic Change Model 



Establishment of the ARSI 
“Systemic Change Model” 

The ARSI model has evolved during the course of 
the project into a truly systemic and successful 
process for school and district reform in science 
and mathematics education. ARSI has come to 
fruition in Year Five, as many of the goals for 
increased local vision, enhanced capacity, and 
access to resources, services and support have 
been met. ARSI has capitalized on the resources, 
people and organizations in the region to develop 
vision, leadership, and commitment to school 
improvement. 

The ARSI plan centered around establishing 
regional Resource Collaboratives to partner with 
participating schools and school districts to 
provide on-site outreach services. Early in the 
Project, it became apparent, that in order to impact 
district level improvement, there needed to be 
support at both the school and district levels. 

ARSI found that in districts successfully 
implementing reform strategies, it was necessary 
to have district-level leadership from the 
instructional supervisor and support from the 
district superintendent. It was also determined that the Teacher Partners’ effectiveness also depended greatly upon support of 
the school principal and district administration. ARSI has continued the development of the District Leadership Team and 
provided training to assist these district teams implement reform strategies during Year 5. 




Over the past two years, ARSI has made a major contribution in education reform in the central Appalachian region through 
the implementation of a its systemic improvement model. The ARSI Model has proved effective in providing both direction 
for school reform and a mechanism for technical assistance to catalyst schools. The model is based on a comprehensive 
review of each participating school’s science and mathematics programs using the Program Improvement Review (PIR) 
process, development of short and long-range plans for program change based on the needs identified, and provision of 
leadership for working the plan through a local leadership team led by a highly skilled and well prepared teacher partner. 

In addition to the structural components outlined, the model contains the following operational features: a) the unit of change 
is the school. Activities occur at the regional, district and classroom levels as well, but they are in support of the changes 
targeted in schools as a whole, b) While the structural components of the model are present in all participating districts, there 
is flexibility in how they are actually implemented. The specifics of how ARSI appears and operates in each school/district 
are guided by the realities of the local context, c) ARSI activities in a particular school/district are developmental and 
tailored to the readiness and ability of the local system to make systemic reforms. As the system develops in its capacity, 
commitment, and capabilities, ARSI’s interactions with the system shift accordingly. 

IL POSITIVE RESULTS ACROSS THE APPALACHIAN STATES SERVED BY ARSI 

Pre-ARSI. At the outset of the project, the following conditions existed: School leaders generally lacked a “vision” of 
quality mathematics and science programs which could provide direction for reform efforts. None of the participating 
schools had a district-wide curriculum in science or mathematics aligned with their state or national standards. Few school or 
district policies existed that promoted excellence in science and mathematics. The financial resources needed to develop 
strong science and mathematics programs were limited and the funds that were available lacked coordination for the greatest 
student benefit. Support from parents and the larger community was lacking. Professional development was usually 
“district based” and focused on generic topics such as improving school discipline or improving student safety in schools. 
Most importantly, student achievement in the ARSI schools lagged well behind other schools in the region and the state. 
Although the school districts in Appalachia differ widely in their “readiness” and ability to participate in significant reform 
efforts, the districts exhibit commonalties that have provided direction for ARSI’s work. 
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The following paragraphs summarize some of the positive aspects of ARSI during Year 5. 



Figure 2--District Capacity for Initiating and Sustaining Reform 



Projected status in 2 Years 



Leadership Development. During year five, ARSI has increased districts* abilities to understand better their own programs 
and the reality of science and mathematics instruction in their districts. As stated by ARSI’s external reviewer, Inverness 
Research Associates, “..we have consistently found that the most important element in determining the ultimate success of a 
reform effort is the presence or absence of skilled ' " ~ ~ " 

and committed leadership. There is no doubt that 
the greatest contribution of ARSI lies in this area: 

ARSI is helping districts identify, train and 
support local leaders who are knowledgeable 
about math and science reform and empowered to 
work towards change in schools and classrooms. ” 

(Appalachian Rural Systemic Initiative: Final 
Report, 2000) ARSI’s efforts in training teacher 
partners, ARSI catalyst school principals, and 
central office administrators (ARSI district 
liaisons) have resulted in a district team that has 
provided extensive leadership for science and 
mathematics program reform efforts. The reforms have occurred primarily at the school level , however, and there is a need to 
build capacity at the district level therefore the leadership developed during ARSI Phase I. Figure 2 shows the ratings of 
selected ARSI districts’ “Current Status” and “ARSI Influence,” from an on-site study conducted by Inverness Research 
Associates using a scale of 1-5 with a “5” representing 100%. 



Current district capacity 



Internal district capacity Pre-ARSI 

ARSI Influence on Improved 
District Capacity 




Figure 3 illustrates the ARSI external evaluator’s assessment of the relationship between the level of “reform leadership” in 
ARSI school districts and the relative influence of ARSI activities on developing this leadership. 



Figure 3 Leadership Capacity in ARSi Districts 
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Strong external political leadership 
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Standards-based Curriculum Development. Since no school district had a fully developed and aligned curriculum at the 
outset of the ARSI project, the development and implementation of aligned, standards-based mathematics and science 
curricula has been a major focus of the project. Resource teachers have participated in curriculum development workshops 
and ARSI curriculum specialists have provided on-site technical assistance to participating districts. Because of ARSI’s 
efforts, participating districts have now developed and implemented a K-12 curriculum aligned with their state’s standards 
for science and/or mathematics. Because of the emphasis on curriculum development and the identified need for resources to 
support the revised program, many ARSI school districts are now selecting and purchasing resources consistent with national 
and state standards. 




Consistent Policies. ARSI has worked with the participating districts’ leadership teams in the development of school and 
district policies, which increase mathematics and science learning opportunities for students. Examples of policies designed 
to increase the vigor and breadth of programs in ARSI districts are: “Requiring algebra as one of three courses in 



mathematics required for graduation” and “increasing the number of science credits required for graduation.” All catalyst 
districts have implemented or revised policies to support mathematics and science programs since the outset of the ARSI 
project. 



Convergence of Resources. The districts participating in the ARSI project reside in some of the poorest school districts in 
the nation. In most cases, schools qualify for “school-wide” Title I assistance and generally have more children receiving 
free-lunch than not. Although, as would be expected, resources for science and mathematics instruction have been limited, 
ARSI has coalesced existing available resources, services, and support for science and mathematics instruction. Local Title I, 
Title II, professional development, and Goals 2000 funds have been applied to ARSI mathematics and science program 
improvement efforts. 



In addition, ARSI has been successful in collaborating with regional agencies and has leveraged an additional 3.4 million in 
resources into the region. The Appalachian Regional Commission, Annenberg Foundation, Eisenhower Regional 
Mathematics/Science Consortium at AEL, Ohio, Kentucky, and Tennessee Departments of Education are some of the 
partners which have provided additional funds for the ARSI project. Further verification of the value of the ARSI model to 
participating districts is the fact that 6 ARSI districts have placed teacher partners in additional schools using local district 
funds in excess of $150,000 during the 1999-2000 school year alone. The information provided in Section IV fully discusses 
the amount and types of external funding that has been reallocated by local districts through ARSI assistance to support 
improving mathematics and science education in ARSI Catalyst Districts. 

Community Engagement. A major problem in most schools, and particularly rural schools in poor communities, is the level 
of parental and community support. Support for ARSI’s vision of high-quality, standards-based mathematics is strong 
among stakeholder groups in the catalyst districts. ARSI community engagement teams include community and business 
leaders as well as parents as role groups. Community engagement teams are operating in 35 ARSI catalyst schools and 
districts. One fifth of the catalyst school principals reported increased business and community involvement and support 
during year five, primarily as a result of ARSI. Community involvement included family mathematics/science/ technology 
nights, community meetings to increase knowledge of science and mathematics instructional programs, assistance with 
student science and/or mathematics projects, community engagement teams, and community/school leadership teams. 

Professional Development. A primary vehicle for science and mathematics program reform in the ARSI districts is 
professional development. Driving this vehicle is the,4&S7 teacher partner, a highly skilled mathematics or science educator 
selected by the catalyst school. During the first four years of the project, teacher partners focused primarily on their own 
classroom and school (catalyst school). During year five, teacher partners began to extend this effort throughout the district. 
The number of teacher partners has increased from 21 in the 1996-97 school year to 66 during the 1999-2000 school year. Of 
this number, eight were released on a full time basis. Local funds are used to pay for part of the release time for many 
teacher partners and every district provided financial support for the teacher partners to attend ARSI professional 
development meetings. Resource Collaboratives have conducted monthly training sessions for teacher partners as well as 
leadership training for principals focusing on mathematics and science program improvement. District liaison and 
superintendent meetings held quarterly also helped insure a strong science and mathematics delivery system in ARSI schools. 



The following graph shows the relative amount of each type of support provided by the ARSI Teacher Partners as reported on 
daily logs. 



Figure 4 Teacher Partners-Time Spent on 
Various Activities 99-2000 
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Improved Student Achievement. Student mathematics and science performance in catalyst schools is improving. Ninety- 
four percent of ARSI catalyst schools are showing improvements in state assessment results in mathematics, science, or both 
since beginning their involvement with ARSI. Analysis of trends in assessment results indicate that ARSI catalyst schools 
are improving relative to comparison schools in non-participating districts and that some gaps with state averages are 
narrowing. 

In examining individual school data, the 
results are even more dramatic. An ARSI 
school that has had a full range of 
interventions in science demonstrates the 
type of results achieved through the project. 

The data shown in Figure 5 are for a school 
in which the ARSI model has been fully 
implemented beginning with a Science 
Program Improvement Review (PIR). Based 
on findings of the PIR, ARSI interventions 
included the development and 
implementation of an aligned, standards- 
based curriculum and extensive staff 
development in inquiry-based instruction. 

Following the ARSI interventions, student 
achievement in science has increased 
significantly and exceeds the state average in 
all assessed areas whereas the student 
achievement in all other content areas, none 



of which had similar interventions, was below the state 
average. 

The data in Figure 6 shows the performance of another ARSI 
school that is equally impressive. This ARSI school is nearing 
“full implementation” meaning that all ARSI standards are 
being addressed. This catalyst school has implemented an 
aligned, standards-based curriculum in mathematics and 
provided extensive problem solving instruction professional 
development for teachers through the ARSI teacher partner. As 
can be seen, the ARSI catalyst school scored above the district 
and state average in every mathematics sub-domain area. 



These data are not unique. In addition to the aggregated 
regional data, assessment data for participating schools and 
districts from each of the six states participating in the ARSI 
project have been analyzed. The results indicate substantial 
improvement in nearly all ARSI schools since the inception of 
the ARSI project. A sample of elementary, middle, and high 
school assessment results from each of the participating states 
is included in results described fully in Section IV. 

Core Data. In addition to the professional development 
activities of teacher partners, ARSI resource collaboratives 
brokered the services of university science and mathematics 
educators and other consultants to provide professional development for participating school districts. During Year 5, over 
5,000 teachers, administrators, parents, and community members are more knowledgeable about standards-based instruction 
as a result of 123,420 participant hours of professional development provided by teacher partners in local districts. In 
addition, ARSI sponsored science and/or mathematics content and pedagogy professional development activities enabled 
3,310 teachers and administrators to be better prepared to teach and support instruction. 

In summary, although many barriers must still be overcome, ARSI has generated an enthusiasm for science and mathematics 
education reform through higher expectations for students, teachers, administrators, and the communities in general. ARSI 
districts have a new vision for and commitment to provide educational opportunities for all students which is being translated 
into higher achievement at all levels of instruction. 



Figure 6 ARSI Catalyst District Mean Scores for Elementary 
— Math 
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SUMMARY OF THE STATUS OF THE ARSI PROJECT AFTER 5 YEARS 



DL 

ARSI has been assisting schools since the 1995-96 school year. Cohort I schools are those that have had involvement with 
ARSI since the 1996-97 school year. Cohort I Catalyst Schools are experiencing significant improvements in the status of 
mathematics and science education that include aligned curricula, standards-based instructional strategies, and inquiry-based 
student learning. This section provides information regarding the changes in mathematics and science programs in schools 
and identifies the districts’ perception of how ARSI has influenced those changes. 

Questionnaire Results for ARSI Cohort I. Each year since 1997, mathematics and science teachers and administrators in 
ARSI districts have completed questionnaires about their attitudes, preparation, classroom practices, and program status. 
Analysis of Spring 2000 questionnaire results and their comparison to baseline results in 1997 provides the following 
information regarding ARSI Cohort I teachers, catalyst schools, and districts: 

Catalyst Teachers report ARSI influence on teaching. Three-fourths of catalyst school teachers (and half of all the 
teachers) report that ARSI has influenced their mathematics and/or science teaching. Teachers who report being influenced 
by ARSI are significantly different from their colleagues, in that they: 

■ hold attitudes more consistent with standards-based approaches; 

■ are better-prepared to implement 
standards-based strategies; 

■ use standards-based strategies in 
their classrooms regularly. 

■ identify fewer barriers to 
implementing their math and 
science teaching 

■ have stronger attitudes consistent 
with standards-based math and 
science 

■ have higher level of preparation 
to use standards-based practices 

■ use technology more frequently 
in support of instruction 



Catalyst school teachers indicate that they 
are better prepared to use the kinds of 
strategies envisioned by ARSI and by state 
and national standards, even though 
overall, implementation of standards-based 
strategies is still in its early stages in the 
catalyst schools. (Figure 7) 



Schools report math/science program improvements. Since becoming involved with ARSI, Cohort I catalyst school 
teachers and administrators report improvements in the school’s mathematics and science programs including: 

■ Better availability of curriculum materials aligned to 
standards. 

■ Better availability of professional development 
appropriate for mathematics and science teachers. 

■ Greater amount of funds allocated to mathematics and 
science. 

■ Better availability of local resource persons to support 
implementation. 

Significant improvement is reported in preparation and practices. 

Specific questionnaire items with statistically significant 
improvement from 1997 to 2000 are included in Figure 8. 

The percentage of Catalyst School Principals that report progress 
in their programs and attribute that progress to ARSI is shown in 
Figure 9. 
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Progress at the district level. During Year 5, ARSI influence and impact has extended beyond the Catalyst Schools. 

District Liaisons report numerous areas of their district- 
wide mathematics and science programs have been 
improved, including: 

* Alignment of district mathematics 

and science curricula 

■ Quality of available instructional 
materials 

■ Availability of professional 
development for mathematics and 
science teachers 

0 District attention focused on 

mathematics and science 

Figure 10 shows district-wide ARSI impact as reported 
by central office personnel. 



IV. ARSI RESULTS— YEAR 5 



A. DRIVER 1: Implementation of Standards- 

Based Mathematics and Science 

Catalyst Schools have shown a significant increase in 
the use of standards-based instructional strategies, use 
of technology, and the understanding and influence of 
national and state standards. Two-thirds of the Catalyst 
School mathematics and science teachers indicate that 
ARSI has influenced their teaching. Furthermore, those 
Catalyst School teachers who report ARSI influence 
have greater levels of: participation in math/science 
professional development; attitudes aligned with 
standards (particularly regarding inquiry-based 
teaching); and use of standards-based instructional 
strategies (e.g., hands-on activities, student-designed 
investigations, cooperative group work, portfolio 
entries, and using technology for data 
collection/analysis). 

As was noted in Section II, Catalyst Schools have made substantial improvements in their mathematics and science curricula, 
aligning them to state assessments and articulating them across grade levels. Improvements are also observed in the quality of 
available instructional materials and the degree of implementation of standards-based instruction. 



Figure 1 1 shows ARSI School/District Implementation Status ratings for the catalyst schools, using a rubric developed by 
ARSI staff. 



Figure 11. District 
Implementation Ratings 
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B. DRIVER 2: Supportive Policies 



The six ARSI states have developed and adopted state standards, curriculum frameworks, and performance expectations that 
are based on and consistent with the national mathematics and science standards. The state standards and other requirements 
form a backdrop that significantly influences local policies and practices. 

Feedback from catalyst school teachers and districts administrators indicates that the following changes in policies impacted 
the school’s and district’s mathematics/science programs: 

• Policies and Procedures to Insure Strategic Planning. Over three-fourths of catalyst school principals attribute to 
ARSI policy changes that increase the amount of time for planning in their school and district focused on improving 
mathematics and science. 



Figure 10-% of District Liaisons Reporting Progress Attributed to 
ARSi 
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Figure 9-% Cohort I Catalyst School principals Reporting 
Progress Attributed to ARSI 
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• Policies and Procedures Insuring Use of Student and School Data. Data-driven planning is a regular occurrence in 
Catalyst Schools. Ninety-four percent analyze state assessment results to identify program needs; 84% include specific 
priorities for mathematics and/or science in school improvement plans (only 38% report doing so prior to their ARSI 
involvement). 

• Policies and Procedures Insuring Adequate Instructional Time . Catalyst schools are addressing issues of time in 
mathematics/science reform, based on Program Improvement Review recommendations. 

Figure 12 indicates the types of policies and the 
percentages of participating ARSI disricts which 
have enacted each in support of the provision of 
higher quality mathematics and science programs. 

C. DRIVER 3: Convergence of Resources 

At the regional level, ARSI has been successful in 
collaborating with other entities to coordinate fiscal 
and human resources in support of standards-based 
mathematics and science and has brought over $3.4 
million in additional resources into the region, 
including: 

• $759,000 from the Annenberg Foundation 
Rural Challenge program supporting activities 
in ten (10) schools, nine (9) of which are in 
ARSI counties. Total funding will be over $1.1 
million and involve 15 ARSI schools. 

• $140,000 through the Appalachian Regional 
Commission (ARC) for technology implementation in 19 ARSI counties from private industry. 

• $215,000 in federal ARC funds to support community engagement, technology, and ARSI Leadership Academy. 

• $430,000 from the Ohio Department of Education to support the Resource Collaborative at Ohio University to serve the 
southeastern Appalachian region of the state. 

• $80,000 from the Eisenhower Regional Math/Science Consortium at AEL for training support for teachers. 

• $165,000 state higher education Eisenhower grants for professional development of mathematics and science teachers in 
Kentucky and Tennessee. 

• $250,000 from the Kentucky Department of Education for Teacher Partners and mathematics/science resources and 
professional development in Kentucky. 

• $950,000 from Goals 2000 funds for five ARSI school districts to improve mathematics and science programs. 

• $350,000 in 1999-2000 GEAR-UP funds to Rockcastle County, Kentucky for middle schools. 

At the local level, funding is still an issue for mathematics and science, but is not seen as a major problem impacting the 
programs. In 1999-2000, Catalyst School Principals and District Liaisons reported a greater diversity of funding sources 
supporting mathematics and science reform, compared to the previous year. Figure 13 represents the funding spent on 
science and math reform reported by participating local districts. 



Figure 13--External Funding Available in ARSI Catalyst Districts 1999-2000 
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Figure 12-Policy Changes Supporting 
Math/Science Improvement 1999-2000 
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In addition to professional development for mathematics and science teachers, Catalyst Schools used these available funds for 
instructional materials and support personnel to assist with mathematics/science implementation. The convergence of resources for 
mathematics and science is attributable, at least in part, to the inclusion of mathematics and/or science in local improvement plans 
developed with ARSI assistance through the consolidated planning process. 



D. DRIVER 4: Broad-Based Support 

Support for ARSI’s vision of high-quality, standards-based mathematics and science is widespread and strong among 
stakeholder groups in the catalyst districts. ARSI specifically includes community leaders, business leaders, and parents as 
role groups on Community Engagement teams, which currently are operating in 35 Catalyst Schools and Districts. One-fifth 
of Catalyst School principals report increased business and community involvement and support over the past year, and most 
attribute the increase to ARSI. 

Districts are developing and implementing programs that involve communities in the support of standards-based mathematics 
and science through a variety of means including support for parents assisting students in science fair projects, community 
meetings to increase involvement in mathematics/science programs, and family mathematics/science/technology nights. 

ARSI Teacher Partners are active in building community support for mathematics and science as well. The types of activities 
included in 1999-2000 were presentations to PTA (18%); family math/science programs (29%); alerting local media about 
math/science events and recognition (53%). 

School and district administrators are key stakeholders for supporting mathematics/science implementation at the Catalyst 
Schools. A majority of teachers perceive that their administrators are knowledgeable about standards-based teaching and are 
very supportive of a hands-on approach to mathematics and science; in fact, the teachers rated administrative support as the 
greatest asset to the mathematics and science programs. For their part, both District Liaisons and Catalyst School Principals 
reported attitudes and perceptions closely aligned with standards-based teaching. The ARSI Summer Leadership Academy 
played a key role in building administrative support. Administrators participating in the Academy demonstrated increased 
knowledge and commitment for supporting mathematics/science reform. On the ARSI-developed implementation rubric, the 
percentage of Catalyst Schools with insufficient administrative knowledge and support decreased from 53% at the beginning 
of ARSI involvement to just 7% at the present time. 

Higher Education Partnerships. ARSFs major partnerships are with the five institutions of higher education where the 
ARSI Resource Collaboratives are located. During Year 5, ARSI has made much progress in institutionalizing these “rural 
science and mathematics education centers” at each university site. The Ohio ARSI Collaborative is incorporated into the 
Regional Professional Development Center at Ohio University and the West Virginia Collaborative is now integrated into the 
College of Education at Marshall University. In Virginia, the ARSI Resource Collaborative is a major component of and the 
operational model for the new Center for Teaching Excellence at the University of Virginia’s College at Wise. The Resource 
Collaborative at the University of Kentucky is now a part of the Appalachian Center and the Tennessee Resource 
Collabortive is in the process of becoming a University of Tennessee “Center” which will more fully integrate its activities 
with other College of Education school district outreach efforts. These relationships provide stability for long-term science 
and mathematics program support for the rural schools in Appalachia. 

Regional Partnerships. ARSI continues to maintain linkages with a number of regional partners. These include: the Rural 
Schools and Community Trust (formerly the Annenberg Rural Challenge), the Appalachian Regional Commission, AEL, the 
Eisenhower Regional Consortium at AEL, Oak Ridge National Laboratory, Project CATS, Forward in the Fifth, and the 
North Central Regional Educational Laboratory. 



State Departments of Education. Partnerships and collaborations with each state’s Department of Education are supporting 
ARSI’s efforts to assist some of the lowest performing districts improve student achievement in mathematics and science. 

State Department of Education collaborations have also generated additional funding to support various aspects of the ARSI 
program, as reported in figure 15. 

Other Collaborations. In addition to the instances cited above, collaboration and networking continue to be a means 
through which Resource Collaboratives provide additional workshop assistance to local schools. ARSI teachers receive 
training as a result of ARSI Resource Collaborative partnerships with state departments of education, the American 
Association for the Advancement of Science (AAAS), the National Nuclear Society, the Woodrow Wilson National 
Fellowship Program, North Carolina State University, Colorado School of Mines, National Science Teachers Association, 
and the Western Carolina University. 
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Driver 5: Student Performance 



Determination of Regional Assessment Results. Because each of the six states in the ARSI region has its own standards 
and assessments, the project faces several challenges in meaningfully representing its impact on regional student 
performance. Particular challenges include aggregating assessment data across states and determining suitable comparison 
indicators. The method used to aggregate the six states’ assessment data is described in Appendix A. 



Aggregated Student Achievement Data. Student achievement 
data for ARSI districts validate the impact of the ARSI model. 
Figure 14 indicates that overall, as determined through an 
aggregation of individual state assessment data, 94% of all 
ARSI Catalyst Schools show improved student performance in 
either mathematics or science. 88% of the Catalyst Schools 
show improvement in science and 90% of the Catalyst Schools 
improved performance in mathematics. These data include all 
schools involved with the ARSI project regardless of the 
number of years in which a teacher partner has been working in 
the district or the level of implementation of the ARSI model. 
The method used to aggregate the six states’ assessment data is 
described in Appendix C. 



Figure 14 ARSI Catalyst Schools Showing Improved Student 
Performance on State Assessments 
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In examining student performance data based on the number of years of participation in the ARSI project, it is interesting to 
note that in Cohort One Catalyst Schools, on which there is data for three years, 91% of the schools show improvement in 
both science and mathematics whereas 9% of the schools show improvement in neither. (Figure 15) 





In Cohort Two Catalyst Schools, having had ARSI involvement for two years, 100% of the schools show improvement in 
either science or mathematics with 82% of the schools showing improvement in mathematics and 88% showing improvement 
in science. (Figure 16) 



Twenty (20) of the forty (40) Catalyst Schools utilizing the “ARSI Model” are rated as “high implementation schools” and 

exhibit the following characteristics: 

• District and school-level leadership and vision to support implementation of standards based mathematics and 
science; 

• Highly skilled Teacher Partner who has provided leadership for school initiated science and mathematics 
program reforms including intensive professional development; 

• Data-driven district and school-level improvement planning for mathematics and science program 
improvement efforts; 

• Mathematics and science curricula aligned to state standards and sequenced across grade levels; 

• Majority of teachers utilize inquiry and/or problem-solving instructional strategies on a regular basis' ; 

• Quality instructional materials aligned with the school’s science and mathematics curricula; 

• Technology utilized in the mathematics and science instructional program and available to access resources supporting 
instruction; 

• Professional development and leadership training based on both personal and school program improvement needs. 

The remaining twenty school districts are at various levels of implementation and exhibit some, but not all, of the 

characteristics identified above. 
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Overall, student achievement in both science and mathematics has improved during the five years of the ARSI project. A 
further analysis of the data, based on the level of implementation of the ARSI Model , indicates that ARSI Catalyst Schools 
nearing “full implementation,” have significantly reduced the gap between their student achievement scores and state 
assessment scores. An even clearer indication of the impact of the ARSI project is the fact that the ARSI high 
implementation schools have improved at a much higher rate than comparison schools in the region. Figures 17 and 18 show 
aggregated student performance levels for science and mathematics in all Catalyst Schools as compared to the high 
implementation Catalyst Schools, aggregated state averages, and comparison school averages. 



Figure 17 Mathematics Performance 
Pre-ARSI (1996) - 1999 
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Figure 18 Science Performance 
Pre-ARSI (1996) - 1999 
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Participating State Student Achievement Data. Individual state student achievement data, based on scores achieved on 
each state’s assessment, provides a clear picture that ARSI is making a positive impact across the region. Figure 19 shows 
the improvement of student performance in selected catalyst schools from 1996 to 1999 in states served by ARSI. Data 
samples from each state have been provided to demonstrate the types of changes occurring. 

Figure 19 Performance of Selected Catalyst Schools 
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Catalyst School/State 


Subject 


1996 


1997 


1998 


199! 


Knott Co. High School/Kentucky 


Math 


13% 


14% 


17% 


21°/ 


Chavies Elementary/Kentucky 


Math 


4% 


7% 


12% 


13°/ 


Clay Co. High School/West Virginia 


Math 




37% 


49% 


56°/ 


Philippi Middle School/West Virginia 


Math 




55% 


57% 


67°/ 


Cocke Co, High School/Tennessee 


Math 


65% 


59% 


61% 


71°/ 


Vinton Co. High School/Ohio 


Math 


23% 


31% 


33% 


36°/ 


Beaver Elementary/Ohio 


Math 


9% 


12% 


30% 


44°/ 


St. Paul High School/Virginia 


Math 


56% 


40% 


63% 


84°/ 


Bath Co. High School/Kentucky 


Science 


2% 


8% 


9% 


13°/ 


Logan Elementary/West Virginia 


Science 




50% 


56% 


59°/ 


Beaver Elementary/Ohio 


Science 


11% 


16% 


32% 


46°/ 


Meigs High School/Ohio 


Science 


27% 


31% 


35% 


42°/ 


St. Paul High School/Virginia 


Science 


69% 




74% 


84°/ 















Ohio. Figure 20 compares the percentage of students in ARSI schools passing the mathematics and science sections of the 
Ohio Proficiency Tests for the years 1997 and 1999. As can be seen, significant improvement was made in mathematics in 8 
out of the 12 catalyst schools whereas 8 of 9 schools made significant gains in science. Ohio ARSI elementary schools made 
significant gains in all but one district (Meigs Eastern) in mathematics. (Figure 21) At the district level, Manchester High 
School, a Cohort 1 Catalyst School, has posted significant improvement in both science and mathematics when compared to 
other Adams County schools. (Figure 22) 
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Figure 20 ARSI Cohort I Schools-Percent Students passing Ohio 
Proficieucy Tests 
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97 


99 


97 


99 


Beaver ES 


4 


11.6 


43.4 


16.3 


45.3 


6 


22.2 


41.9 


18.5 


30.2 


Eastern ES 


4 


NR 


44.6 


NR 


43.9 


6 


NR 


62.5 


NR 


51.8 


McConnelsville ES 


4 


41.9 


40.8 


28.8 


55.3 


6 


48.8 


66.7 


38.1 


43.5 


Syracuse ES 


4 


26.9 


40 


19.2 


40 


6 


71.4 


33.3 


33.3 


40.7 


Vinton Co. HS 


9 


61.5 


63.4 


NA 


75.8 


12 


46.2 


35.5 


45.6 


38.7 


Meigs HS 


9 


63.8 


70.6 


NA 


69.2 


12 


18.1 


28.6 


37 


41.3 


UC 


9 


51.3 


37.3 


NA 


52.6 






Figure 22. Comparisou of ARSI Catalyst School (Manchester H.S.) with Other Adams County 
High Schools 
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Manchester HS 


40.2% 


46.4% 


51.3% 


59.0% 


45.7% 


65.4% 


37.7% 


52.6% 


65.1% 


73.8% 


North Adams HS 


58.5% 


44.3% 


57.0% 


46.1% 


60.7% 


60.7% 


56.7% 


58.9% 


73.6% 


72.2% 


Peebles HS 


46.6% 


30.6% 


47.3% 


51.6% 


41.4% 


61.2% 


56.1% 


71.9% 


36.4% 


60.2% 


West Union HS 


40.7% 


36.8% 


46.6% 


36.2% 


42.6% 


55.6% 


47.2% 


54.4% 


64.3% 


71.4% 



Virginia Students in Virginia Catalyst Schools were among the Top Improving Schools on the spring 99 SOL assessments. 
Figure 23 shows the performance of Cohort I Schools — those participating in ARSI activities for three years. All grades 
tested in all schools show improvement in mathematics from 1998 to 2000 whereas 5 of 7 grades tested in science show 
improved performance during this two year period. 

Figure 24 shows the percentage of students passing the Virginia Standards of Learning (SOL) Tests at grade 8. All ARSI 
middle schools show gains in both science and mathematics. Districts having strong support for ARSI from their 
administration and have successful Teacher Partners are showing the greatest gains in the region. 



Figure 23 Cohort I Student Performance in Virginia Catalyst Schools 




Grade 


Content 


1998 


2000 


Clintwood ES 


3 


Math 


34.21 


71.43 


Clintwood ES 


3 


Science 


28.95 


75 


Clintwood ES 


5 


Math 


44.83 


75 


Clintwood ES 


5 


Science 


44.83 


61.11 


Clintwood ES 


8 


Math 


51.43 


65.57 


Clintwood ES 


8 


Science 


82.35 


80.33 


St. Paul HS 


HS 


Math 


63.01 


93.21 


St. Paul HS 


HS 


Science 


74 


88.83 


St. Paul HS 


8 


Math 


4 


61.9 


St. Paul HS 


8 


Science 


2.04 


100 


Powell Valley MS 


5 


Math 


51.09 


57.55 


Powell Valley MS 


5 


Science 


54.41 


52.83 


Powell Valley MS 


8 


Math 


32.05 


65.47 


Powell Valley MS 


8 


Science 


57.05 


88.24 






West Virginia . Students in West Virginia are considered “proficient” if they score above the 50 th percentile on the SAT-9. 
To determine progress over time, the student assessment data for West Virginia has been aggregated to compare movement 
of K-12 students from the lower assessment Quartiles to the 3 rd and 4 th Quartiles. Figures 25 and 26 compares the percentage 
of students scoring in the 3 rd and 4 h Quartiles in 1997 and 1999 in mathematics and science. These data show a significant 
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improvement in the percentage of students scoring at the upper two Quartiles and also indicates the ARSI schools are 
“closing the gap” between themselves and the state average. 




Kentucky Eleventh grade students in ARSI districts in Kentucky scored below both the state and regional averages in both 
science and mathematics at the outset of the ARSI project. Figures 27 and 28 demonstrate the changes in student 
performance recorded for the period 1996 through 1999. In both mathematics and science, ARSI student achievement scores 
have improved more than the regional comparison districts and the gap between ARSI district scores and the state average 
has been significantly reduced. 



Figure 27. Grade 11 Mathematics 
Performance Comparison for Kentucky 
1996-1999 
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Figure 28. Grade 11 Science Performance 
Comparison for Kentucky 1996-1999 
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North Carolina Only two districts in North Carolina are eligible for the ARSI project, Swain and Graham Counties. In both 
counties, the ARSI Catalyst School is the county High School. Figure 29 and 30 shows the improved performance on end-of- 
course exams for Algebra I and Biology. Students in both high schools posted improved scores from 1996-1999 and 
Robbinsville High exceeded the state average on the Algebra I exam while both Robbinsville and Swain High Schools 
exceeded the state average on the Biology exam in 1999. 



Figure 29. North Carolina End of Course 
Exam Comparison for Algebra I 1997-1999 
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Figure 30. North Carolina End of Course 
Exam Comparison for Biology 1997-1999 
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Tennessee The state of Tennessee utilizes “value added gains” as one measure of school achievement. “The minimum value 
or expectation is that students will gain a year’s average growth (compared to the national norm) for a year’s instruction in 
each subject area. This is expressed as a 100% gain. Higher or lower performance is rated accordingly.” (From; Tennessee 
Department of Education, “How to Interpret the Report Card) Figures 31 and 32 show the comparison of mathematics and 
science assessments for 1996 to 1999 in ARSI districts. As can be seen, student performance in mathematics exceeded the 
expectation in four of five districts and student performance in science exceeded the expectation in all five districts. 



Student achievement for a single ARSI district is shown in Appendix A. 



Figure 31 . Comparison TN Valu e -Add e d Mathematics 
Assessment: Percent of National Norm Gain In ARSI 
Catatyst Districts 1996-1999 
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F. Driver 6: Student Equity Issues 



Kentucky ARSI district mathematics achievement scores for 1999 were disaggregated to determine if differences in 
performance existed based on gender. The disaggregated sample, which represented students at all grades tested, included 
1291 females and 1217 males. As can be seen in Figure 33, fewer females score at the novice level (lowest quartile) on the 
Commonwealth Accountability Testing System student assessment in mathematics at all age levels. A higher percentage of 
female students also achieved a “proficiency rating” on mathematics at each grade level. 



Figure 33. Comparison of Females and Males Scoring at the “Novice” and “Proficient” Levels on the Commonwealth 
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Middle School (8 th Grade) 


High School (11 “ Grade) 


Level of Performance 


Novice 


Proficient 


Novice 


Proficient 


Novice 


Proficient 


Females 


212% 


28.6% 


25.6% 


23.7% 


23.1% 


26.6% 


Males 


24.5% 


25.7% 


29.0% 


21.4% 


27.7% 


22.5% 



In comparing these findings with ARSI core data for all six states, it is interesting to the note the following correlations 
(Figure 34); 

■ Females make up a higher percentage of students than males in lower level math courses represented (courses not 
higher than Algebra 1 and Geometry). 

■ Females make up a higher percentage of students than males in higher level mathematics courses (Algebra II and up). 

■ A significantly higher percentage of females than males scored a “C” or higher in all mathematics courses taken 
although a slightly higher percentage of males scored a “C” or higher in calculus (a course normally taken after the 
11 th grade assessment). 



Figure 34. Male and Female Mathematics Course Enrollment and Achievement in ARSI Schools based on Core Data 
for 23 School Districts in Six States 



Enrollment/ Achievement Data 


Females 


Males 


Number/Percentage of Students Enrolled in Lower Level Math Courses 


3602 (52.5%) 


3161 (47.5%) 


Number/Percentage of Students Enrolled in Higher Level Math Courses 


1189 (54.6%) 


987 (45.4%) 


Number/Percentage of Students Achieving a “C” or Higher on Lower Level Math Course 


2139 (57.2%) 


1598 (42.8%) 


Number/Percentage of Students Achieving a “C” or Higher on Higher Math Course 


1912 (54.1%) 


1625(45.9%) 


Number/Percentage of Students Achieving a “C” or Higher in Calculus 


84 (48.3%) 


90 (51.7%) 



These data indicate that there is a need for ARSI schools to examine the curriculum, instructional methodologies, resources, 
course-taking patterns and other learning opportunities as well as cultural and community expectations to determine why 
there is an observed difference in performance based on gender. 



Kentucky ARSI district science achievement scores for 1999 were also disaggregated to determine if differences in 
performance existed based on gender. The disaggregated sample, which represented students at all grades tested, included 
594 females and 670 males. As can be seen in Figure 35, the distribution of males and females was similar for each category 
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on the Commonwealth Accountability Testing System student assessment in science at both the elementary and middle 
school levels. At the high school level, however, a significantly higher percentage of female students scored at the novice 
level whereas a corresponding higher percentage of males achieved a “proficiency” rating. 



Figure 35. Comparison of Females and Males Scoring at the “Novice” and “Proficient” Levels on the Commonwealth 





Elementary School (4 th Grade) 


Middle School (7 th Grade) 


High School (11 '"Grade) 


Level of Performance 


Novice 


Proficient 


Novice 


Proficient 


Novice 


Proficient 


Females 


39.6% 


93% 


99.2% 


0.8% 


20.7% 


19.8% 


Males 


41.7% 


93% 


992% 


0.8% 


27.9% 


31.6% 



In comparing these findings with ARSI core data for all six states, it is interesting to the note the following correlations 
(Figure 36). 

■ Enrollment in lower level science courses is slightly higher among females. 

■ Enrollment in higher level science courses is slightly higher among males and significantly more males were enrolled 
in Physics. 

■ A significantly higher percentage of females earned a grade of “C” or higher in lower level science courses, whereas 
a greater percentage of males earned a “C” or higher in the higher level science course, and a significantly higher 
percentage of males earned a “C” or higher in physics (more than a 3/2 ratio). 



Figure 36. Male and Female Science Course Enrollment and Achievement in ARSI Schools based on Core Data for 23 
School Districts in Six States 



Enrollment/ Achievement Data 


Females 


Males 


Number/Percentage of Students Enrolled in Lower Level Science Courses 


2848 (51.2%) 


2609 (47.8%) 


Number/percentage of Students Enrolled in Higher Level Science Courses 


589 (49.6%) 


599 (52.4%) 


Number/Percentage of Students Achieving a “C” or Higher on Lower Level Science 
Courses 


1898 (55.5%) 


1523 (44.5%) 


Number/Percentage of Students Achieving a “C” or Higher on Higher Level Science 
Courses 


439 (47.8%) 


479 (52.2%) 


Number/Percentage of Students Taking Physics 


209 (43.5%) 


271 (56.5%) 


Number/Percentage of Students Achieving a “C” or Higher in Physics 


137 (39.6%) 


209 (60.4%) 



Science enrollment and achievement patterns for males and females in the six ARSI states are consistent with the Kentucky 
assessment data indicating higher student performance by males at the secondary level. These data indicate that there is a 
need for ARSI schools to examine the learning opportunities afforded girls in higher level science courses. The overall 
curriculum, instructional methodologies, resources, and course-taking patterns as well as cultural and community 
expectations need to be examined to determine why there is an apparent gender difference in both enrollment and 
performance in higher level science courses. 

SUMMARY 

The ARSI model has proven to be a successful reform strategy for the Appalachian region. As a result of ARSI’s efforts: 

■ A corps of science and mathematics leaders has been established. 

■ Student achievement has increased in science and/or mathematics in the Appalachian region served by the ARSI 
project. 

■ The quantity and quality of science and mathematics learning opportunities for students have increased. 

■ Regional collaboration and support for science and mathematics reform efforts has improved. 

The primary goal of establishing “a sustainable system to support standards-based science and mathematics instruction” is 
nearing fruition and should be realized during Phase II of the ARSI project. 
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